Since the year 2003 the police department of san francisco has been reporting crime data. Of particular intererest for analysis is the different crime types, the time of incident (both date but also time of day down to the minute) but also the coordinates of the incident (given in latitude/longitude). From this data its possible to look at temporal and spatial trends of different crimes over the last 20+ years. The different categoris of crimes include Vehicle theft, vandalism, robbery, prostiution and many more. I have choosen to look at the trends of vehicle thefts since the trend in many ways are unique compared to the other types of crimes, but i will go more into detail later.
The temporal trend of vehicle thefts
The first super relevant thing is to look at how the number of incidents of vehicle theft have evolved over time.
Code
import pandas as pd import numpy as npimport matplotlib.pyplot as pltdata=pd.read_csv("C:/NoterDTU/6_semester/Social_data/website_2/merged_data.csv")crimes = data[['Category', 'Year']]crimes = crimes[(crimes['Category']=='VEHICLE THEFT') & (crimes['Year']!=2025) ]crime_counts = crimes["Year"].value_counts().sort_index()crime_counts.plot(kind="bar",color="indigo",edgecolor="black")plt.ylabel("Number of incidents")plt.xlabel("Year")plt.title("Number of Vehicle thefts per year (2003-2024)")plt.show()
One thing one notices almost immedialty is the sudden drop from 2005 to 2006 and onwards. In 2005 the numbers peak at around 17.500 vehicle thefts while the next years it drops by around 10.000 and remains in that range going forward. This approximately 60% of the crimes that just stopped happening in one year. That seems quite strange. Some sources suggest that the fact that cars are hard to break into and harder to dissamble might be 😀
We will later compare vehicle theft to some other crimes in order to see if this was the overall trend of crime data (spoilers its not)
The spatial trends
One thing one might suspect could explain the sudden drop is if there suddenly where a focus from the police deparment on a specific area. As previously mentioned they have the data for where the different crimes are happening and one would then think that they then priotize forces in these specific areas. In order to look at the we plot a time series of where the incidents of vehecle theft is happening for a given month and try to see if there is any difference.
Code
import pandas as pdimport foliumfrom folium.plugins import HeatMapWithTimefrom IPython.display import display# Load datadf = pd.read_csv("C:/NoterDTU/6_semester/Social_data/website_2/merged_data.csv")# Filter for vehicle thefts between 2003-2007df_filtered = df[(df['Category'] =='VEHICLE THEFT') & (df['Year'].between(2003, 2024))].copy()# Extract relevant columns and drop NAdf_filtered = df_filtered[['Latitude', 'Longitude', 'Month', 'Year']].dropna()# Check for valid coordinatesvalid_coords = df_filtered[ (df_filtered['Latitude'].between(-90, 90)) & (df_filtered['Longitude'].between(-180, 180))]# Define month mapping and ordermonth_mapping = {"January": 1, "February": 2, "March": 3, "April": 4, "May": 5, "June": 6, "July": 7, "August": 8, "September": 9, "October": 10, "November": 11, "December": 12}month_names =list(month_mapping.keys())# Create numerical month columndf_filtered['MonthNum'] = df_filtered['Month'].map(month_mapping)# Sort by year and monthdf_filtered = df_filtered.sort_values(['Year', 'MonthNum'])# Prepare heat data and time indexheat_data = []time_index = []for year inrange(2003, 2025):for month_num inrange(1, 13): month_data = df_filtered[ (df_filtered['Year'] == year) & (df_filtered['MonthNum'] == month_num) ] coords = month_data[['Latitude', 'Longitude']].values.tolist() heat_data.append(coords) time_index.append(f"{month_names[month_num-1]}{year}")# Only create map if we have data# Create base mapbase_map = folium.Map(location=[37.75800, -122.41914], zoom_start=11.5,zoom_control=0,scrollWheelZoom=False,dragging=0)# Add heatmap with timeHeatMapWithTime( heat_data, index=time_index, # Time labels showing month and year auto_play=0, max_opacity=0.5, radius=11, min_opacity=0.1, gradient={0.2: 'blue', 0.4: 'lime', 0.6: 'orange', 0.8: 'red'}, display_index=True, use_local_extrema=1, name="Vehicle Thefts", blur=1).add_to(base_map)# Display mapdisplay(base_map)
Make this Notebook Trusted to load map: File -> Trust Notebook
From the heatmap its clear that a lot of the incidents happen in the north-eastern/eastern part of san francisco. And that trend does not change as teh years go by. But its clear that there is fewer crimes compared to 2003-2005 and the years after since the point of the heat map are more spread out.
The news article https://www.eastbaytimes.com/2007/02/16/car-thefts-decrease-statewide/ also tells this story Where the general trend for vehicle theft are on the decline. The reason behind this trend is both the fact that more and more vehicle have implemented alarms, key-coding systems. But also there has also been set up 16 autu-theft task forces. There have also been an increase in the use of so called “bait-cars” which are used as bait to track down the drivers and since its normal that they steal more than one car the number of cars that are being stolen drops significantly. In 2006 they made 357 arrest with the use of bait-cars. Which might have severely impacted the amount of cars stolen
Last thing that could be interesting to look at is if the trends in data are unique to car theft or if the over all trend of crime is the “same” as with the car theft.
Correlation between crimes
In order to compare the crimes. We choose to look at how correlated the different crimes are. What we are comparing is the amount of crimes for a given month example burgraly versus vehicle theft in the month of january 2015. We can the make an scatter plot and compute how related the data are. The scatter plot might also show other trends. But we will get to that. 😎
